We describe and analyze an algorithm for predicting a sequence of n-dimensional binary vectors based on a set of experts making vector predictions in [0, 1]. We measure the loss of individual predictions by the 2-norm between the actual outcome vector and the prediction. The loss of an expert is then the sum of the losses experienced on individual trials. We obtain bounds for the loss of our ex...

Modular learning, inspired by divide and conquer, learns a large number of localized simple concepts (classiiers or function approximators) as against single complex global concept. As a result, modular learning systems are eecient in learning and eeective in generalization. In this work, a general model for modular learning systems is proposed whereby, specialization and localization is induce...

Dropout is a very effective method in preventing overfitting and has become the go-to regularizer for multi-layer neural networks recent years. Hierarchical mixture of experts hierarchically gated model that defines soft decision tree where leaves correspond to nodes gating models softly choose between its children, as such, hierarchical partitioning input space. In this work, we propose varian...

Machine comprehension of visual information from images and videos by neural networks suffers two limitations: (1) the computational inference gap in vision language to accurately determine which object a given agent acts on then represent it language, (2) shortcoming stability generalization classifier trained single, monolithic network. To address these limitations, we propose MoE-VRD, novel ...

A three-level hierarchical mixture model for classiŽcation is presented that models the following data generation process: (1) the data are generated by a Žnite number of sources (clusters), and (2) the generation mechanism of each source assumes the existence of individual internal class-labeled sources (subclusters of the external cluster). The model estimates the posterior probability of cla...

A three-level hierarchical mixture model for classification is presented that models the following data generation process: (1) the data are generated by a finite number of sources (clusters), and (2) the generation mechanism of each source assumes the existence of individual internal class-labeled sources (subclusters of the external cluster). The model estimates the posterior probability of c...

This paper presents the results of Persian handwritten word recognition based on Mixture of Experts technique. In the basic form of ME the problem space is automatically divided into several subspaces for the experts, and the outputs of experts are combined by a gating network. In our proposed model, we used Mixture of Experts Multi Layered Perceptrons with Momentum term, in the classification ...

A mixture of experts models the conditional density a response variable using finite regression with covariate-dependent weights. We extend model by allowing parameters in both components and weights to evolve time following random walk processes. Inference for time-varying richly parameterized is challenging. propose sequential Monte Carlo algorithm online inference based on tailored proposal ...